SENTENCE CLASSIFICATION

Syntactically Aware Neural Architectures for Definition Extraction (N18-2)

Challenge:	Recent studies show that attention-based models benefit from more focused attention over local regions.
Approach:	They propose a syntax-aware local attention which restrains attention over syntactically relevant words.
Outcome:	The proposed model performs better on all benchmark datasets, including sentence classification and sequence labeling tasks.

Copied to clipboard

Luis Espinosa-Anke, Steven Schockaert

DMix: Adaptive Distance-aware Interpolative Mixup (2022.acl-short)

Challenge:	Existing approaches to identify definitional knowledge in text corpora are based on Wikipedia-like definitions.
Approach:	They propose to combine Convolutional and Recurrent Neural Networks to train definitional knowledge in text corpora.
Outcome:	The proposed models can be applied to more noisy domain-specific corpora.

Copied to clipboard

Ramit Sawhney, Megh Thakkar, Shrey Pandit, Ritesh Soun, Di Jin, Diyi Yang, Lucie Flek

Challenge:	Interpolation-based regularisation methods such as Mixup have shown to be effective for various tasks and modalities.
Approach:	They propose an adaptive distance-aware interpolative Mixup that selects samples based on their diversity in the embedding space.
Outcome:	The proposed method achieves state-of-the-art on sentence classification over existing methods on 8 benchmark datasets across English, Arabic, Turkish, and Hindi languages while achieving benchmark F1 scores in 3 times less number of iterations.

A Deep Neural Network Sentence Level Classification Method with Context Information (D18-1)

Copied to clipboard

Xingyi Song, Johann Petrak, Angus Roberts

Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment (2025.acl-long)

Challenge:	Existing methods that use context for sentence classification are difficult to scale . Usually, sentences are treated as separate instances for the task . however, in many situations the sentence that is the focus of classification appears in a context that can provide additional information.
Approach:	They propose a method that uses potentially large contexts to classify sentences . they use an LSTM, and short-span features to classize sentences based on a stacked CNN .
Outcome:	The proposed method consistently improves on two different datasets.

Copied to clipboard

Yongxin Huang, Kexin Wang, Goran Glavaš, Iryna Gurevych

Retrofitting Contextualized Word Embeddings with Paraphrases (D19-1)

Challenge:	Multilingual sentence encoders are often trained to map sentences from different languages into a shared semantic vector space . cross-lingual alignment training distorts optimal monolingual structure of semantic spaces of individual languages . a modular solution can be used for cross-linguistic tasks such as cross-language semantic similarity and zero-shot transfer .
Approach:	They propose a modular training system that embeds sentences from different languages into a shared semantic vector space.
Outcome:	The proposed solution achieves better performance across all tasks compared to monolithic models.

Copied to clipboard

Weijia Shi, Muhao Chen, Pei Zhou, Kai-Wei Chang

Representation Learning

Challenge:	Contextualized word embeddings can be useful for downstream applications, but they can be over-sensitive to contexts.
Approach:	They propose a method to retrofit contextualized word embeddings with paraphrases to minimize the variance of word representations on paraphrased contexts.
Outcome:	The proposed method improves on sentence classification and inference tasks.

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (2021.emnlp-main)

Copied to clipboard

Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang, Saksham Singhal, Xian-Ling Mao, Heyan Huang, Xia Song, Furu Wei

Multilinguality

Challenge:	Multilingual T5 pretrains a sequence-to-sequence model on monolingual texts, but it has shown promising results on many cross-lingual tasks.
Approach:	They propose a partially non-autoregressive objective for text-to-text pre-training and propose mT6 to improve cross-lingual transferability over multilingual T5.
Outcome:	The proposed model improves cross-lingual transferability over existing models.

Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders (2026.findings-eacl)

Copied to clipboard

Mathis Le Bail, Jérémie Dentan, Davide Buscaldi, Vanier Sonia

Multi-Granularity Guided Fusion-in-Decoder (2024.findings-naacl)

Challenge:	Concept-based explanations for large language models are not well understood in text classification.
Approach:	They propose a model with a specialized classifier head and activation rate sparsity loss for sentence classification . they compare it to existing models with HI-Concept and ConceptShap .
Outcome:	The proposed model improves both the causality and interpretability of the extracted features.

Copied to clipboard

Eunseong Choi, Hyeri Lee, Jongwuk Lee

Reconstruction Attack on Instance Encoding for Language Understanding (2021.emnlp-main)

Challenge:	Open-domain question answering requires deriving factual responses without explicit evidence . recent approaches combine retrieval of relevant information with response generation .
Approach:	They propose a model that concatenates multiple contexts in the decoding phase . they propose MGFiD, which harmonizes passage re-ranking with sentence classification .
Outcome:	The proposed model outperforms existing models on Natural Questions and TriviaQA datasets . it aggregates evident sentences into an anchor vector that instructs the decoder .

Copied to clipboard

Shangyu Xie, Yuan Hong

Responsible & Trustworthy NLP

Challenge:	Existing private learning schemes which protect data privacy can be used to train models using instance encoding.
Approach:	They propose to recover the private training data and use it to break a private learning scheme TextHide.
Outcome:	The proposed attack would advance privacy-preserving machine learning in the context of natural language processing.

Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers (2023.eacl-main)

Copied to clipboard

William Held, Diyi Yang

Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024.findings-naacl)

Challenge:	Multilingual transformer-based models demonstrate remarkable zero and few-shot transfer across languages by learning and reusing language-agnostic features.
Approach:	They propose to prune attention heads from a fixed model to reduce interference as a model acquires more languages.
Outcome:	The proposed model can reduce interference by pruning language-specific attention heads.

Copied to clipboard

Weimin Lyu, Xiao Lin, Songzhu Zheng, Lu Pang, Haibin Ling, Susmit Jha, Chao Chen

Robustness in NLP

Challenge:	Existing methods for textual backdoor detection are task-specific and less effective beyond sentence classification.
Approach:	They propose a task-agnostic method for backdoor detection that leverages final layer logits and an efficient pooling technique.
Outcome:	TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional methods.

What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)

Copied to clipboard

Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, Marco Baroni

WYWEB: A NLP Evaluation Benchmark For Classical Chinese (2023.findings-acl)

Challenge:	a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques.
Approach:	They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways .
Outcome:	The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them.

Copied to clipboard

Bo Zhou, Qianglong Chen, Tianyu Wang, Xiaomi Zhong, Yin Zhang

Challenge:	Existing benchmarks for classical Chinese are inadequate to evaluate performance of different NLP models.
Approach:	They propose an evaluation benchmark for classical Chinese NLP, which evaluates existing models.
Outcome:	The proposed benchmark evaluates the performance of existing models in classical Chinese.

One-Dimensional Object Detection for Streaming Text Segmentation of Meeting Dialogue (2025.findings-acl)

Copied to clipboard

Rui He, Zhongqing Wang, Minjie Qiang, Hongling Wang, Yifan.zhang Yifan.zhang, Hua Xu, Shuai Fan, Guodong Zhou

Natural Language Interfaces

Challenge:	Current text segmentation models exhibit numerous limitations, such as imbalances in labels that affect the stability of model training and discrepancies between the model’s training tasks (sentence classification) and the actual text segmenting.
Approach:	They implement a sliding window-based segmentation method and employ two different levels of sliding window based balanced label strategies to stabilize the training process of the streaming segmentation model.
Outcome:	The proposed method is robust, controllable, and achieves state-of-the-art performance.

Locality Preserving Sentence Encoding (2021.findings-emnlp)

Copied to clipboard

Changrong Min, Yonghe Chu, Liang Yang, Bo Xu, Hongfei Lin

Revisiting the Negative Data of Distantly Supervised Relation Extraction (2021.acl-long)

Challenge:	Existing word embeddings can be used to learn sentence embedds on the sentence level.
Approach:	They propose a sentence embedding method that uses the inner product to compute semantic similarity between sentences.
Outcome:	The proposed method encodes sentences better in the sense of semantic structures.

Copied to clipboard

Chenhao Xie, Jiaqing Liang, Jingping Liu, Chengsong Huang, Wenhao Huang, Yanghua Xiao

Explicit Alignment Objectives for Multilingual Bidirectional Encoders (2021.naacl-main)

Challenge:	Existing methods for relation extraction with distant supervision generate plenty of training samples but noisy labels and imbalanced training data cause problems.
Approach:	They propose a method that automatically labels a sentence with relational triples from a knowledge base.
Outcome:	The proposed method outperforms existing methods even with false positive samples.

Copied to clipboard

Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, Graham Neubig

Syntactic Text Processing

Challenge:	Pre-trained cross-lingual encoders have proven impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resourced languages.
Approach:	They propose a method to align multilingual encoders using two explicit alignment objectives that align the multilingual representations at different granularities.
Outcome:	The proposed method achieves gains of up to 1.1 average F1 score on sequence tagging and 27.3 average accuracy on retrieval over the XLM-R-large model.

A Set Prediction Network For Extractive Summarization (2023.findings-acl)

Copied to clipboard

Xiaoxia Cheng, Yongliang Shen, Weiming Lu

A Dual-Attention Network for Joint Named Entity Recognition and Sentence Classification of Adverse Drug Events (2020.findings-emnlp)

Challenge:	Recent approaches to extracting salient sentences from source document are naive and lack dependencies between sentences.
Approach:	They propose a set prediction network to detect redundancy relationship between sentences . they use a non-autoregressive decoder to predict sentences in parallel .
Outcome:	The proposed method outperforms previous state-of-the-art models on extracted summary datasets.

Copied to clipboard

Susmitha Wunnava, Xiao Qin, Tabassum Kakar, Xiangnan Kong, Elke Rundensteiner

SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain (2021.naacl-main)

Challenge:	Adverse drug events (ADEs) are a leading cause of death in the United States and cost around $30 $130 billion every year.
Approach:	They propose a multi-grained joint deep network to learn ADE entity recognition and ADE sentence classification tasks.
Outcome:	The proposed model improves state-of-art F1 score on the MADE 1.0 benchmark of EHR notes.

Copied to clipboard

Ruochen Zhang, Carsten Eickhoff

Challenge:	Existing methods for state tracking are limited and state changes are less densely distributed over utterances.
Approach:	They propose to turn to simplified, fully observable systems that show some of these properties.
Outcome:	The proposed system shows that state changes occur infrequently while messages are "chatter" it allows for rich descriptions of state while avoiding the complexities of other settings.

Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification (2020.emnlp-main)

Copied to clipboard

Prithviraj Sen, Marina Danilevsky, Yunyao Li, Siddhartha Brahma, Matthias Boehm, Laura Chiticariu, Rajasekar Krishnamurthy

Explainability & Interpretability in NLP

Challenge:	Existing approaches to explain models are difficult to interpret and have undesirable biases.
Approach:	They propose a neural network architecture for learning transparent sentences . they use linguistic expressions built on top of predicates extracted using shallow natural language understanding .
Outcome:	The proposed model outperforms statistical relational learning and other neuro-symbolic methods and performs better than black-box recurrent neural networks.

Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts (D18-1)

Copied to clipboard

Di Jin, Peter Szolovits

SciBERT: A Pretrained Language Model for Scientific Text (D19-1)

Challenge:	Existing sentences classification models often classify sentences in isolation without considering the context in which sentences appear.
Approach:	They propose a hierarchical sequential labeling network to make use of contextual information within surrounding sentences to help classify the current sentence.
Outcome:	The proposed model outperforms the state-of-the-art methods by 2%-3% on two benchmarking datasets for sequential sentence classification in medical scientific abstracts.

Copied to clipboard

Iz Beltagy, Kyle Lo, Arman Cohan

Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violence (2021.findings-acl)

Challenge:	SciBERT is a pretrained language model based on BERT to improve performance on scientific NLP tasks.
Approach:	They propose a pretrained language model based on BERT to improve NLP performance . they evaluate on sequence tagging, sentence classification and dependency parsing .
Outcome:	The proposed model improves on sequence tagging, sentence classification and dependency parsing tasks with datasets from a variety of scientific domains.

Copied to clipboard

Andrew Halterman, Katherine Keith, Sheikh Sarwar, Brendan O’Connor

Challenge:	a new corpus-level evaluation approach for event extraction is needed in social science applications . human annotations are often required to extract the actions of political actors and actors . a novel corpus evaluation approach can guide creation of similar social science-oriented resources .
Approach:	They propose a corpus-based approach to event extraction that integrates corpus evaluation with real-world social science . they use human annotations to read and label every document for mentions of police activity events .
Outcome:	The proposed method can guide creation of similar social-science-oriented resources in the future.

Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)

Copied to clipboard

Lifu Tu, Caiming Xiong, Yingbo Zhou

Counterfactual Adversarial Learning with Representation Interpolation (2021.findings-emnlp)

Challenge:	Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
Approach:	They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning .
Outcome:	The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters.

Copied to clipboard

Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, Rong Zhang

Robustness in NLP

Challenge:	Existing models with statistical bias are prone to memorized correlations . large pre-trained models such as BERT have revolutionized the model development paradigm in natural language processing .
Approach:	They propose a framework to tackle the problem from a causal perspective using a latent space interpolation approach.
Outcome:	Extensive experiments show that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.

PcMSP: A Dataset for Scientific Action Graphs Extraction from Polycrystalline Materials Synthesis Procedure Text (2022.findings-emnlp)

Copied to clipboard

Xianjun Yang, Ya Zhuo, Julia Zuo, Xinlu Zhang, Stephen Wilson, Linda Petzold

MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification (2020.emnlp-main)

Challenge:	305 open access scientific articles are used for synthesis action graphs . lack of annotated data has hindered progress in this field .
Approach:	They propose to annotate Polycrystalline Materials Synthesis Procedures PcMSP from 305 open access scientific articles for the construction of synthesis action graphs.
Outcome:	The proposed dataset contains the synthesis sentences, entity mentions and intra-sentence relations extracted from the experimental paragraphs.

Copied to clipboard

Qianli Ma, Zhenxi Lin, Jiangyue Yan, Zipeng Chen, Liuhong Yu

Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs (2023.findings-emnlp)

Challenge:	Existing models for sentence classification use linear convolution, which may not be sufficient to model the non-consecutive dependency of the phrase and may overfit the sequential information.
Approach:	They propose a model that extracts multi-scale n-gram features for understanding the semantic meaning of sentences by some key-phrases located at different positions.
Outcome:	The proposed model outperforms existing models on eight benchmark datasets and is competitive against state-of-the-art models.

Copied to clipboard

Yihong Liu, Haotian Ye, Leonie Weissweiler, Renhao Pei, Hinrich Schuetze